Rethinking the Expressive Power of GNNs via Graph Biconnectivity

Designing expressive Graph Neural Networks (GNNs) is a central topic in learning graph-structured data. While numerous approaches have been proposed to improve GNNs in terms of the Weisfeiler-Lehman (WL) test, generally there is still a lack of deep understanding of what additional power they can systematically and provably gain. In this paper, we take a fundamentally different perspective to study the expressive power of GNNs beyond the WL test. Specifically, we introduce a novel class of expressivity metrics via graph biconnectivity and highlight their importance in both theory and practice. As biconnectivity can be easily calculated using simple algorithms that have linear computational costs, it is natural to expect that popular GNNs can learn it easily as well. However, after a thorough review of prior GNN architectures, we surprisingly find that most of them are not expressive for any of these metrics. The only exception is the ESAN framework (Bevilacqua et al., 2022), for which we give a theoretical justification of its power. We proceed to introduce a principled and more efficient approach, called the Generalized Distance Weisfeiler-Lehman (GD-WL), which is provably expressive for all biconnectivity metrics. Practically, we show GD-WL can be implemented by a Transformer-like architecture that preserves expressiveness and enjoys full parallelizability. A set of experiments on both synthetic and real datasets demonstrates that our approach can consistently outperform prior GNN architectures.Comment: ICLR 2023 notable top-5%; 58 pages, 11 figure

Your Transformer May Not be as Powerful as You Expect

Relative Positional Encoding (RPE), which encodes the relative distance between any pair of tokens, is one of the most successful modifications to the original Transformer. As far as we know, theoretical understanding of the RPE-based Transformers is largely unexplored. In this work, we mathematically analyze the power of RPE-based Transformers regarding whether the model is capable of approximating any continuous sequence-to-sequence functions. One may naturally assume the answer is in the affirmative -- RPE-based Transformers are universal function approximators. However, we present a negative result by showing there exist continuous sequence-to-sequence functions that RPE-based Transformers cannot approximate no matter how deep and wide the neural network is. One key reason lies in that most RPEs are placed in the softmax attention that always generates a right stochastic matrix. This restricts the network from capturing positional information in the RPEs and limits its capacity. To overcome the problem and make the model more powerful, we first present sufficient conditions for RPE-based Transformers to achieve universal function approximation. With the theoretical guidance, we develop a novel attention module, called Universal RPE-based (URPE) Attention, which satisfies the conditions. Therefore, the corresponding URPE-based Transformers become universal function approximators. Extensive experiments covering typical architectures and tasks demonstrate that our model is parameter-efficient and can achieve superior performance to strong baselines in a wide range of applications. The code will be made publicly available at https://github.com/lsj2408/URPE.Comment: 22 pages; NeurIPS 2022, Camera Ready Versio

One Transformer Can Understand Both 2D & 3D Molecular Data

Unlike vision and language data which usually has a unique format, molecules can naturally be characterized using different chemical formulations. One can view a molecule as a 2D graph or define it as a collection of atoms located in a 3D space. For molecular representation learning, most previous works designed neural networks only for a particular data format, making the learned models likely to fail for other data formats. We believe a general-purpose neural network model for chemistry should be able to handle molecular tasks across data modalities. To achieve this goal, in this work, we develop a novel Transformer-based Molecular model called Transformer-M, which can take molecular data of 2D or 3D formats as input and generate meaningful semantic representations. Using the standard Transformer as the backbone architecture, Transformer-M develops two separated channels to encode 2D and 3D structural information and incorporate them with the atom features in the network modules. When the input data is in a particular format, the corresponding channel will be activated, and the other will be disabled. By training on 2D and 3D molecular data with properly designed supervised signals, Transformer-M automatically learns to leverage knowledge from different data modalities and correctly capture the representations. We conducted extensive experiments for Transformer-M. All empirical results show that Transformer-M can simultaneously achieve strong performance on 2D and 3D tasks, suggesting its broad applicability. The code and models will be made publicly available at https://github.com/lsj2408/Transformer-M.Comment: 20 pages; ICLR 2023, Camera Ready Version; Code: https://github.com/lsj2408/Transformer-

Genome-wide eQTLs and heritability for gene expression traits in unrelated individuals

BACKGROUND: While the possible sources underlying the so-called ‘missing heritability’ evident in current genome-wide association studies (GWAS) of complex traits have been actively pursued in recent years, resolving this mystery remains a challenging task. Studying heritability of genome-wide gene expression traits can shed light on the goal of understanding the relationship between phenotype and genotype. Here we used microarray gene expression measurements of lymphoblastoid cell lines and genome-wide SNP genotype data from 210 HapMap individuals to examine the heritability of gene expression traits. RESULTS: Heritability levels for expression of 10,720 genes were estimated by applying variance component model analyses and 1,043 expression quantitative loci (eQTLs) were detected. Our results indicate that gene expression traits display a bimodal distribution of heritability, one peak close to 0% and the other summit approaching 100%. Such a pattern of the within-population variability of gene expression heritability is common among different HapMap populations of unrelated individuals but different from that obtained in the CEU and YRI trio samples. Higher heritability levels are shown by housekeeping genes and genes associated with cis eQTLs. Both cis and trans eQTLs make comparable cumulative contributions to the heritability. Finally, we modelled gene-gene interactions (epistasis) for genes with multiple eQTLs and revealed that epistasis was not prevailing in all genes but made a substantial contribution in explaining total heritability for some genes analysed. CONCLUSIONS: We utilised a mixed effect model analysis for estimating genetic components from population based samples. On basis of analyses of genome-wide gene expression from four HapMap populations, we demonstrated detailed exploitation of the distribution of genetic heritabilities for expression traits from different populations, and highlighted the importance of studying interaction at the gene expression level as an important source of variation underlying missing heritability. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/1471-2164-15-13) contains supplementary material, which is available to authorized users

ENSO combination mode and its influence on seasonal precipitation over southern China simulated by ECHAM5/MPI-OM

Recent studies show that a combination mode (C-mode) represents the seasonally modulated dynamics of ENSO, which plays an important role in maintaining the western North Pacific anomalous anticyclone. This C-mode could obviously influence the East Asian climate, especially since the contribution of ENSO to southern China’s precipitation has weakened since the late 1990s. This paper evaluates whether the C-mode and its influences on precipitation over southern China can be realistically described by the climate model ECHAM5/MPI-OM. The authors find that the model is able to reproduce the spatial and temporal variability of the C-mode and the asymmetric responses of air–sea variations in the tropical Pacific. The model also reveals the observed significant effects of the C-mode on the wintertime and springtime rainfall over southern China during El Niño events. The findings have implications for ECHAM5/MPI-OM being a valuable tool for simulating and predicting the C-mode-related seasonal precipitation over southern China

Preparation of Salen–Metal Complexes (Metal = Co or Ni) Intercalated ZnCr-LDHs and Their Photocatalytic Degradation of Rhodamine B

Salen–metal complexes (SalenM) were successfully intercalated into ZnCr layered double hydroxides (LDHs) through coprecipitation method, then a series of novel organic–inorganic hybrid materials were obtained. The structure and properties of the materials were thoroughly characterized by inductively-coupled plasma atomic emission spectrometry (ICP-AES), powder X-ray diffraction (XRD), Fourier transform infrared spectrometry (FTIR), scanning electron microscopy (SEM), and ultraviolet visible diffuse reflectance spectroscopy (UV-Vis DRS). Meanwhile, with Rhodamine B (RhB) as a target contaminant, the photocatalytic activities of SalenM-intercalated ZnCr-LDHs were investigated and compared with the traditional LDHs (ZnCr-LDHs, ZnCoCr-LDHs, and ZnNiCr-LDHs). Furthermore, the effect of the intercalation amount of SalenM (M = Co or Ni) on the photocatalytic activity was studied. The results showed that when the molar ratio of SalenM to Cr was 0.75, SalenM-intercalated ZnCr-LDHs exhibited significantly higher photocatalytic activities than the traditional LDHs. The degradation rates of RhB reached about 90%, and all of them had good recycling rates. In addition, the kinetics of photocatalytic process and the mechanism of photocatalysis are discussed

Inhibitory Activity and Mechanism of Action with Thymol against the Blueberry Pathogenic Fungi Caused by <em>Neopestalotiopsis clavispora</em>

Decay caused by Neopestalotiopsis clavispora is an important postharvest disease of blueberries that seriously affects the commercial value of blueberry fruit. In this paper, we studied the inhibitory activity and mode of action of thymol against the pathogenic fungus of blueberries caused by Neopestalotiopsis clavispora. The results demonstrated that thymol administration could limit mycelial growth in vitro; the inhibitory effect was positively connected with thymol mass concentrations, and the minimal inhibitory concentration (MIC) was 100 mg/L. Further investigations revealed that MIC thymol treatment dramatically reduced the germination of pathogenic spores and led to an increase in the conductivity of the pathogen, leakage of contents, and a decrease in pH. Propidium iodide (PI) staining experiments demonstrated that MIC thymol caused damage to mycelial cell membranes. Additionally, MIC thymol treatment promoted mycelium malondialdehyde content accumulation, inhibited superoxide dismutase (SOD) and catalase (CAT) enzyme activities, decreased adenosine triphosphate (ATP), adenosine diphosphate (ADP), and adenosine monophosphate (AMP) content and energy charge levels, and the fluorescence intensity of mycelium caused by MIC thymol treatment was significantly increased by the 2,7-Dichlorodi-hydrofluorescein diacetate (DCFH-DA) assay. The results of this study indicate that thymol suppresses the proliferation of Neopestalotiopsis clavispora by compromising the integrity of their cell membranes, promoting the accumulation of cellular reactive oxygen species (ROS), and interfering with energy metabolism